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D uring the past year, we have repeatedly heard 
that the nation’s public schools are facing a high school 
completion crisis. My research at the Urban Institute, for 
example, reveals an overall graduation rate of 68 
percent. Even more troubling, there’s only a fifty-fifty 
chance for a student from a historically disadvantaged 
minority group to finish, the same odds as flipping a coin 
(Swanson 2004). But I am not alone in this assessment. 
Findings from independent studies conducted at a 
variety of institutions — Johns Hopkins University, Boston 
College, the Manhattan Institute, and others — all point in 
a similar direction (Balfanz and Legters 2004; Greene 
and Foster 2003; Haney et al. 2004). Far too many of 
our youth, particularly poor and minority students, are 
failing to complete high school with a diploma. 

At the same time, the media has publicized cases where 
school systems apparently encourage underperforming 
high school students to leave or use administrative 
sleight-of-hand and other suspect methods to cover up 
the dropout problem. While these stories are certainly 
intriguing and disconcerting, we do not know how wide- 
spread such practices are. These controversies over 
graduation rates and the No Child Left Behind Act 
(NCLB), however, helped push high school reform to the 
forefront of the educational issues highlighted by the 
2004 presidential campaigns. Will the solutions touted 
actually be put into practice? Only time will tell. 

All too often, when complex issues of social and 
economic importance collide with policy and politics, 
heat is generated but little light. In particular, it may be 
difficult for local educators to parse the rhetoric from the 
reality and to figure out what this all means for their 
schools and students. This article attempts to provide 



some practical insight into NCLB and its implications for 
graduation rates and to highlight some issues of 
particular relevance to school administrators. I also 
hope that this piece will point to areas where local 
school leaders may take increasing ownership of the 
search to find solutions for the high school completion 
crisis. 



T he basics of NCLB and graduation rates 

Depending on whom you ask, we have the No Child Left 
Behind Act (NCLB) either to thank or to blame for the 
unpleasant discovery of low graduation rates. As we all 
know by now, NCLB holds public schools accountable 
for student performance and does so in a way that 
considerably extends the reach of previous federal 
legislation into local education systems. It is also a well- 
known fact that the law requires achievement test 
scores to be the primary means of measuring student 
performance. However, the state accountability systems 
mandated under NCLB must also include at least one 
additional academic outcome. At the high school level, 
this “Other Academic Indicator” must be the graduation 
rate. 

No Child Left Behind defines high school graduation 
rates in a particular way. Namely, only students who 
receive a regular standards-based diploma on time with 
their class should be counted as high school graduates. 
States must use this definition for purposes of NCLB 
accountability, though it may be inconsistent with 
common sense understanding of what constitutes a 
“high school graduate” or with other definitions that have 
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been used in the past. In fact, states have challenged 
aspects of this federal definition, including whether 
students receiving certificates of attendance and GEDs 
or those who take longer than four years to finish high 
school should be counted as graduates under NCLB. 

It is also worth considering why the law requires 
accountability over both test scores and graduation 
rates in the first place. Suppose that an accountability 
system attached stakes only to test scores. One way for 
high schools to boost test scores would be to push the 
lowest performing students out of school. This kind of 
“gaming” strategy would result in higher achievement 
scores and would, therefore, help schools to avoid 
sanctions. But these apparent gains would be artificial, 
attained only at the high cost of creating more dropouts. 
Requiring accountability for graduation rates is intended 
to counteract this type of perverse incentive that could 
undermine the spirit of the law. 

Nowhere in the NCLB legislation does it state that 
graduation rates should be treated any differently than 
test scores with respect to accountability. However, the 
federal regulatory process gives states a tremendous 
amount of flexibility when implementing provisions 
related to graduation rates, latitude that does not exist 
for test scores. To be clear, some degree of state 
independence is not necessarily a bad thing. Such state 
autonomy might offer some advantages for 
implementing a complex law like NCLB and tailoring its 
provisions to meet local needs. But in this case, weak 
federal oversight has seriously jeopardized the law's 
intended safeguard against “push-out.” Although there 
are many potential areas for concern here, I discuss two 
major issues below — the way graduation rates are 
calculated and the stakes attached to those results. 



w here do graduation rates come from? 

States have the authority to decide what assessments 
they will use to measure student achievement for NCLB 
(subject to federal approval, of course). Similarly, states 
can choose their own formula for calculating graduation 
rates under the law. A variety of different approaches 
have been adopted toward measuring performance in 



both of these areas. Yet, while theory and practice offer 
reasonably well-defined criteria for assessing the quality 
of standardized tests, no such consensus exists with 
respect to measuring graduation rates. 

In a recent Urban Institute study, I reviewed the set of 
state plans outlining the accountability systems being 
implemented under NCLB (Swanson 2003). My 
particular focus was on the ways that states are 
calculating graduation rates. If we were to map out 
these approaches, we would find a veritable patchwork 
quilt. In fact, several states are even being allowed to 
use dropout rates in their accountability systems despite 
the law’s stipulation of graduation, not dropout, rates. 
More troubling is that different formulas for computing a 
graduation rate can generate much different results. 

How large are these disparities? In another recent study 
I calculated graduation rates for a large number of 
school districts using four different formulas similar to 
those states are using under NCLB. The results reveal 
average disparities of as much as 14 percent between 
the results produced by the various formulas. I have 
also found a clear distinction between formulas based 
primarily on enrollment data and those that incorporate 
data on dropouts. Comparatively speaking, the latter 
systematically overestimate the graduation rate. It is 
worth noting that this is exactly what we would expect to 
find if large numbers of dropouts were going uncounted, 
as researchers have long suspected. Since the majority 
of states use such dropout-dependent graduation rate 
formulas, we may be viewing the performance of many 
school systems through a distorted, rose-colored lens. 



A 

double standard for accountability? 

The two overarching goals of NCLB are to raise overall 
performance levels and to close gaps between highl- 
and low-performing groups. Analyses of federal 
regulations and state implementation by the Urban 
Institute and the Civil Rights Project at Harvard 
University have revealed a troubling double standard in 
NCLB accountability (Swanson 2003; Orfield et al. 
2004). While standards are uniform and rigorous for test 
scores, they are nothing of the kind for graduation rates. 
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For test scores, all states must adopt the same high 
performance target. This is the well-known goal of 
having 100 percent of students reaching proficiency by 
the 2013-2014 school year. States must also set explicit 
year-by-year goals for progress, with repeated failure to 
meet those goals carrying strong consequences for 
schools. To promote narrowing of achievement gaps, 
schools cannot make “adequate yearly progress” (AYP) 
under the law unless they meet their annual 
performance objective for students as a whole and for 
each subgroup defined on the basis of race and 
ethnicity, socioeconomic level, disability status, and 
English language proficiency. These are certainly high 
standards, perhaps even too high as some critics of the 
law have argued. But these lofty ambitions only apply to 
tested achievement. 

Weak federal regulations on graduation accountability 
have opened a door that permits states to opt for a 
lower road. In fact, most states have seen that opening 
and are now taking a path of less resistance. States are 
allowed to establish their own target levels for 
graduation rates. In practice, goals range anywhere 
from 50 to 100 percent of students finishing high school. 
Further, in most states any amount of improvement in 
graduation rates can be considered adequate 
progress — even if annual gains are barely measurable 
and do not meet a reasonable minimum threshold for 
performance. So a school might be able to make AYP if 
its graduation rate creeps up from an abysmal 25.1 
percent to a nearly-as-abysmal 25.2 percent. Finally, 
state accountability systems are not required to consider 
subgroup graduation rates when making their primary 
determinations about rewards and sanctions under the 
No Child Left Behind Act. All told, a mere four states 
took the high road of requiring both a firm floor for 
graduation rates and also disaggregating results for 
subgroups when determining adequate yearly progress. 



H ow does this affect schools? 

Some critics of NCLB have argued that we should 
postpone attaching any stakes to graduation rates until 
we are able to figure out the best way to calculate these 
statistics. To use an old expression, the horse is out of 
the barn when it comes to NCLB and graduation rates. 
So the time has passed for this wait-and-see argument 



to carry much sway. In any event, there is a legitimate 
counterargument that we should not let the perfect be 
the enemy of the good. Holding schools accountable for 
graduation rates (even if imperfectly) serves an 
important purpose under the law — safeguarding against 
raising test scores on the backs of dropouts. 

Imperfect graduation rate statistics should be a reason 
for concern. For the most part, however, relatively little 
attention has been devoted to developing a better 
understanding of the practical benefits and limitations 
associated with various approaches to calculating 
graduation rates. At this juncture, we understand that a 
given formula may produce systematically biased 
results. We also have some sense of the particular 
factors that may complicate attempts to accurately 
measure graduation rates. These include things like 
transfer and migration of students into and out of school 
systems; retention in grade; and general difficulties 
associated with documenting students who dropout. 

Unfortunately, the impact of transfer, retention, and 
undercounting dropouts on the accuracy of graduation 
rates has been difficult to gauge. There are a variety of 
reasons for this intractability. Many school systems do 
not attempt to systematically collect data about these 
processes. Even if an attempt is made, these 
phenomena are themselves hard to measure 
accurately. Further, dynamics like transfer and grade 
retention may impact certain calculation methods or 
formulas to different degrees. As a result, the question 
of which graduation rate formula is “best” (i.e., produces 
the most accurate result) may depend heavily on local 
conditions. 

Allowing every school district or school to decide on its 
own formula for calculating graduation rates might 
arguably provide for a good matching between methods 
and local circumstances. But this would also render 
meaningless any semblance of the uniformity that is 
necessary for meaningful accountability over this 
important outcome. On the other hand, I would also be 
hesitant to recommend that principals rely entirely on 
state-generated statistics when trying to understand the 
process of high school completion. As suggested above, 
official state figures may be misleading if they use 
flawed methods. In addition, state estimates may tell 
only part of the story and fail to illuminate the 
educational processes that contribute most to low on- 
time graduation rates. For instance, an excessive grade 
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retention rate may be the leading culprit in one school 
while dropout plagues another. 

Many schools have at their disposal much more detailed 
information about students, their performance, and their 
progress through high school than the state either 
possesses or takes into account when calculating 
graduation rates. The examples below illustrate how 
school-level data may help principals to move beyond 
state statistics. A more proactive and critical approach 
will help administrators to more clearly understand the 
particular challenges facing their schools and even to 
better recognize progress when and where it is being 
made. 



Figure 1 : High School Completion Patterns for a Hypothetical School 




Graduates Year 4 



Ninth Graders Year 1 

Revised “Cohort” Rate 

Graduates Year 4 



(Ninth Graders Year 1) - (Out Transfers) + (In Transfers) 

Timely Progress Rate (TPR) 

(Graduates Year 4)+(On-Time Transfers) 

Ninth Graders Year 1 



70.0% 



87.5% 



85.0% 



^3oing the math ... An example 

To illustrate the value of locally derived knowledge, 
consider the following hypothetical scenario. Suppose 
that a state uses a very simple formula to calculate 
graduation rates: dividing the number of graduates in a 
particular year by the number of ninth graders four years 
earlier. (Although this is a very rudimentary statistic, it is 
similar to indicators actually being used under NCLB. It 
can also produce an accurate estimate of the true 
graduation rate in some circumstances, although not 
under the conditions proposed in this example.) Now, let 
us suppose that a particular school maintains detailed 
records about its students that are far more in-depth 
than the data collected through the centralized state 
information system. This school-level information might 
include, among other things, grade retention and 
promotion histories; reliable counts of dropouts; records 
of transfers into and out of the school; and 
documentation indicating whether out-transfers were 
progressing normally at the time they left. 

As the figure shows, this school starts with an entering 
class of 100 ninth graders and, four years later, 70 
students receive diplomas. According to the state's 
formula, this would produce a graduation rate of 70 
percent. A closer examination of the school’s own 
detailed records, however, reveals a much more 
complicated picture. 

Over this four-year period, it seems that there has been 
a large amount of transfer out of the school, perhaps 



due to a local economic downturn. Twenty students 
have moved out but none have moved into the school. 
By contrast, there have been only 10 dropouts. With this 
information, the school could compute a revised “cohort” 
graduation rate by removing the students who left the 
ninth grade cohort from the denominator of the 
calculation. Those transfer students would no longer be 
eligible to receive a diploma from (or dropout of) this 
high school. Making this adjustment, we find the 
considerably higher graduation rate of 87.5 percent. 

At first glance, the state data seemed to point towards a 
severe dropout problem. Based on the school’s more 
subtle insights, however, we instead discovered 
dramatic out-migration of students with a relatively 
modest amount of dropout. 



T hinking further outside the box ... 
extending the example 

NCLB calls for attention to every child. But in our 
example, one fifth of the ninth graders who attended the 
hypothetical school (i.e., the 20 out-transfers) did not 
count towards the school’s revised graduation rate in 
any way. A graduation rate based on the cohort of 
students who remain in a school does provide an 
accurate perspective on school performance in many 
situations. But how meaningful would that statistic be in 
a school where the student population is highly 
transitory and only a small fraction of those entering 
ninth grade actually finished their high school career in 
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that same school? When the group of students for 
whom a school is accountable is extremely fluid, we 
may need to think further outside the box when trying to 
understand high school completion. 

We can expand on the example above by calculating 
something that we might call a Timely Progress Rate or 
TPR. This TPR indicator has the advantage of counting 
all 100 students who attend the school in the 
denominator. Students will appear in the numerator if 
they either (1) graduate on time with their class or (2) 
remain on schedule for graduation at the time they 
transfer out of the school. When we calculate this third 
indicator, we find a rate of 85 percent — slightly lower 
than the revised cohort rate but still much higher than 
the state’s official figure. A comparison of these three 
rates together offers a deeper insight than can be 
afforded by any of the three taken alone. We might 
interpret the findings of our hypothetical analysis as 
suggesting that while this school is losing students to 
transfer, it is doing a reasonable job of bringing students 
through the high school grades and to graduation. 

The lesson to be taken away here is not that one 
statistic is necessarily right and others are wrong, or 
even that one rate is “righter” than another. The cohort 
rate and the timely progress indicator, for instance, are 
both valid statistics related to the process of high school 
completion. However, they measure somewhat different 
things and therefore produce somewhat different 
results. Instead, we should recognize that examining 
graduation rates from multiple local perspectives can 
help schools to better identify the extent of a potential 
problem, its possible causes and, perhaps more 
importantly, promising solutions. This type of information 
may also be a valuable tool in principals’ efforts to share 
their school’s experiences with the local community and 
to advocate to important stakeholders and decision- 
makers on behalf of their students' interests. 



c 

losing thoughts on moving toward 
smarter accountability 

Educational accountability systems, if they are to 
function properly, must include all affected parties in a 
meaningful way and establish clear expectations and 
responsibilities for each respective agent. Thoughtful, 



smarter accountability must be about more than just 
sticks and carrots. It must be about arming schools with 
the information they need to provide students with the 
opportunity to achieve their fullest potential. 

Accountability must evolve beyond its current punitive 
spirit to become relentlessly and constructively focused 
on providing children with the supports and services 
they need to succeed. Only when educational 

accountability becomes a true partnership among 
federal, state, and local stakeholders will it be able to 
serve its intended purpose, improving the education and 
lives of our nation’s youth 

With the advent of NCLB, graduation rate accountability 
has become the educational law of the land, a small part 
of it at least. Some uncertainty continues to persist 
around key issues, like exactly how to calculate 
graduation rates and how much weight should be 
placed on graduation versus test scores when attaching 
stakes to performance. What there can be little doubt of, 
however, is that finishing high school is critically 
important for the lives of our students and our 
communities. We know that youth who fail to earn a 
high school diploma suffer both economically and 
socially, and that they will find many opportunities for 
further advancement closed. Likewise, poorly educated 
communities experience disproportionately high rates of 
unemployment, crime, incarceration, and dependency 
on public aid. 

Evidence increasingly points to the existence of a 
broad-based high school completion crisis in this 
country. When nearly one-third of all public school 
students are failing to graduate with a diploma, it 
becomes difficult to argue that this is still someone 
else’s problem or that it is isolated just in the very 
lowest-performing schools. It is in our own backyards. 

Principals have an important role to play in leading local 
reform efforts and in turning around struggling schools. 
But as this article points out, local wisdom and vigilance 
can also help us keep accountability honest. Principals 
are in a critical position to assure that the official 
statistics driving educational accountability, at the very 
least, accurately reflect the reality of life in America’s 
schools. Ideally, we must also make sure that high 
stakes accountability leads to meaningful solutions for 
our most pressing problems, not just more punishment 
for our most troubled schools. 
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